Hmm one thing you might want to be careful of is his afternoon sleep, If you don't get him down in a certain time period then maybe drop it for the day as with him getting up at 4pm after the sleep hes probably not tired again for some time.